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© Pattern recognition device. 

© Data which represents a border line of a region of a video image to be observed and which contains large 
luminance difference compared with that of a defect to be detected is excluded in judgement when tt belongs to 
a predetermined region (PT 31) so that it is possible to detect minute defect or defects reliably even when a 
position of a workpiece is displaced from that of a master or when the border line of the workpiece is not 
coincident with that of the master due to a difference in size therebetween. 
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PATTERN RECOGNITION DEVICE 



BACKGROUND OF THE INVENTION 

The present invention relates to a pattern recognition device and, particularly, to such device useful in 
recognizing minute defects on a printed circuit board, a coating of an automobile or a lens, etc, by means 
5 of a television camera. 

A proposal is disclosed in Japanese Patent Application Laid-open No. 39581/1985, in which an 
existence of an article or defects in various portions thereof is recognized by deciding whether or not a 
signal level of a portion of a video signal of the article which corresponds to an area of the article to be 
monitored is within a predetermined range. In detail, luminance of the video signal corresponding to the 
io monitor region of the article is compared with luminance of a video signal of a corresponding region of a 
standard article ( referred to as "master" hereinafter ) having a normal contour and it is decided that the 
monitor region contains a defect when there is a difference in luminance therebetween. 

In a case where a minute defect or defects existing in a wide monitor region of such as printed circuit 
board, an automobile coating, a lens, etc., are to be detected, it is necessary to detect reliably even a very 
rs small variation of luminance of the video signal thereof which is not included in the signal from the master. 
Since an area of such defect may be considerably smaller than that of the monitor region, it is difficult, 
practically, to derive a video signal portion corresponding to the defect from a video signal of the monitor 
region when there is a portion or portions, such as curved, angled or patterned portion which may reflect 
illumination light considerably locally causing a wide and considerable variation of luminance to be 
20 produced regardless of defect, included in the monitor region, which may constitute a background noise 
large enough to make the intended video signal portion inseparable therefrom. This fact is very severe 
when the portion in question is a boarder line of the monitor region. 

When the latter is the case and -when the video signal of the monitor region is displaced in position 
relatively to that of the master or the monitor region is different in size from the master region, a luminance 
25 information obtained from such displaced boarder region becomes considerably larger than that of the 
minute defect even if such positional displacement is within an acceptable range and thus it becomes 
impossible to distinguish the defect therefrom. 

Since, therefore, in order to detect such minute defect precisely, it is necessary to compare respective 
pixels of the monitor region with those of the master region one by one successively. In this case, the 
30 problem of the positional displacement between the monitor and the master regions becomes very severe 
causing this approach to be not practical. 



SUMMARY OF THE INVENTION 

35 

An object of the present invention is to provide a pattern recognition device capable of detecting a 
minute defect included in a wide region without influence of portions in the wide region in which large 
difference in luminance is included. 

According to the present invention, the above object of the present invention is achieved by a provision 
40 of a pattern recognition device in which, in comparing a first video image data obtained by a TV camera 
from a monitor region with a second video image data obtained by the TV camera from a master, image 
data portions concerning boarder areas of them are excluded. 



45 BRIEF DESCRIPTION OF THE DRAWING 

Figs. 1 to 4 illustrate a principle of the present pattern recognition device; 

Fig. 5 is a block circuit diagram showing an embodiment of the present pattern recognition device; 

and 

so Fig. 6 is a flow-chart showing an operation of the present pattern recognition device. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

Before describing an embodiment of the present invention, it may be helpful in fully understanding the 
present invention to describe a principle of the present invention. Figs. 1 to 4 illustrate the principle, in 
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which Fig. 1 shows an original image PTO having an area defined by a boarder BDO. Luminance of the 
original image PTO is assumed as 1 and an outside area of the image is indicated as having luminance 0. 

Fig. 2 shows shift images PT01 and PT03 obtained by shifting the original image PTO in Fig. 1 
vertically oppositely by one pixel, respectively, and PT02 and PT04 obtained by shifting the original image 
s PTO horizontally oppositely by one pixel, respectively. Luminance informations of a desired number ( in this 
case, four ) of such shifted images are overlapped as shown in Fig. 2 and a sum of luminance informations 
for each pixel area is obtained. An average luminance of an overlapped image PT1 is obtained according to 
the following equation: 

PT1 = (PT01 + PT02 + PT03 + PT0*)/4 (1) 
io Then, the luminance information of the overlapped image PT1 is reduced by the original image PTO 
and an absolute value of a resultant difference, which is given below, is assigned to each of the pixels 
constituting the respective image portions, resulting in a difference image PT2 as shown in Fig. 3. 

PT2 = | PTO - PT1 | (2) ...ox, 
Thus, only the boarder information added by the overlapping processing shown in Fig. 2 is left as it is. 
is Thereafter, a threshold level of luminance is set to a predetermined value TH according to the 
luminance information of the difference image PT2 in such a manner as shown below 
TH = 1/4 (3) 

and, then, pixels of the difference image PT2 which have luminance equal to or higher than the set level are 

assigned to a logic level "1 M when 
20 PT2 £ TH (4) 

as shown by 

PT3 = "1" (5) 

and to a logic level "0 ff when 

PT2 < TH (6) 
25 as shown by 

PT3 = "0° (7) 

to define a boundary excluded image PT3 as shown in Fig. 4. That is, a boundary excluded area PT31 of 
the boundary excluded image PT3 to which logic level w 1" is assigned is excluded and a comparison with 
the master is performed only for an area PT32 to which logic level "0* is assigned. 
30 Therefore, so long as a sample to be monitored on defect is located such that its boarder area is fallen 
in the area PT31. a defect of similar size to one pixel existing in the area PT32 can be reliably detected by 
the comparison with the master having no such defect. 

The operation of the equation (1) can be represented generally as follows: 

35 mm 

l Z f(X+4X, y'+AY) 

jo-a i°-n . . . ( 8 ) 

f(X,Y) = " 

(2m + 1) • (2n + 1) 
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where AX and AY are shift amount data in X and Y directions and represented by 
A X = i. A x (i = -n- +n) (9) 
and 

A Y = j. A y (j = -m - +m) (10) 

where A x and A y are unit shifts of image in X and Y directions and ffxTTrepresents an integrate of a 
luminance f(X,Y) at a position (X.Y) on the image plane. Thus, it is possible to obtain mean luminance of 
pixels constituting the respective image portions by overlapping (2m + 1) (2n + 1) shifted images. It 
should be noted that the operation of the equation (8) is equivalent to obtain a smoothed image by filtering 
so the original image PTO (Fig. i ) through a spatial low-pass filter. 

The comparison of the threshold level TH with the luminance of the difference image PT2 can be 
represented as follows: 
|f(X,Y) ■ fTXYfl S TH (11) 
|f(X,Y) • OTl < TH (12) 
ss Fig 5 shows, in block diagram, a general construction of an embodiment of the pattern recognition 
device according to the present invention. In Fig. 5, a video image signal VD of a workpiece 2 such as lens 
to be checked on defect is obtained by a TV camera 3. which is converted by an A/D converter 4 into a 
digital data of a predetermined number of bits. e.g.. 8 bits. The digital data is stored in an input image 
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memory 5 comprising a frame memory. 

The video signal VD is also supplied to a timing control portion 6 directly and to a monitor portion 8 
through a superimpose circuit 7. 

The timing control portion 6 generates a clock signal in synchronism with a horizontal and vertical 
s synchronizing signal portions of the video signal VD, forms an address data ADR in response to an 
instruction signal COM from a CPU 10 through a CPU bus 11 and sends it to a video address bus 12 so 
that address data can be sent to a plurality of image memories 25 to 27 connected to the video address 
bus 12 pixel by pixel. 

The image data written in the input image memory 5 are read out sequentially according to correction 
io address data ADRX obtained in an address correction circuit 13 on the basis of the address data ADR from 
the timing control portion 6 and trasferred through a video data bus 14. That is, the address correction 
circuit 13 corrects the address data ADR according to a correction data D A m supplied from the CPU 10 by 
amounts corresponding to the shift amounts AX and AY and the resultant correction address data ADRX is 
sent to the input image memory 5 so that image data of the shifted images PT01 to PT04 previously 
is mentioned with respect to Fig. 2 can be sent to the video data bus 14 under the control of the CPU 10. 

The CPU 10 responds to an instruction signal from an operation input portion 21 to execute, with an aid 
of a RAM 23, a pattern recognition processing program according to a program data stored in a ROM 22. 

The working image memory 25 connected to the data bus 14 comprises a frame memory of, for 
example, 16 bits and functions to store a luminance information of the overlap image PT1 obtained on the 
20 basis of luminance information of the original image PTO as mentioned with reference to Figs. 2 and 3. The 
working memory 25 further functions to store a luminance data of the difference image PT2 obtained on the 
basis of the overlap image PT1 and the original image PTO as mentioned with reference to Fig. 3 and an 
image data of the workpiece. 

A comparison data memory 28 also connected to the data bus 14 stores a result of comparison of the 
25 image data of the workpiece with the image of the master image of the workpiece. 

A boundary data memory 29 also connected to the data bus 14 stores data of the boundary excluded 
image PT3 obtained with respect to the master image as mentioned with reference to Fig. 4. 

A boundary exclusion area data memory 30 also connected to the video data bus 1 4 functions to store 
a pattern data generated by a graphic display controller 31, under control of the CPU 10. When the latter 
30 controls the graphic display controller 31 according to an instruction data from the operation input portion 
21. the image data in the boundary exclusion area data memory 30 is displayed through a D/A converter 32 
and the superimpose circuit 7 on the monitor portion 8 such that data of the area to be excluded on test, 
i.e., area of the image having large variation of luminance while not to be considered as a final product, can 
be stored selectively in the boundary exclusion area memory 30 mannually. 
35 A data operation portion 35 connected to the video data bus 14 executes the operations of the 
boundary data mentioned with respect to the equations (1) to (10). 

That is, in performing the summing operation mentioned with respect to the equations (1) and (8) or the 
subtracting operation shown by the equations (2) and (9), data to be added or subtracted is latched in a first 
and a second latch circuits 36 and 37 and the aimed operation is performed in an addition/subtraction 
40 circuit 38 under control of the CPU 10. 

In performing the division operation of the equations (1) and (8), data to be divided is put in the second 
latch circuit 37 and shifted by a shifter 39 toward least significant bit. 

In addition thereto, in performing the comparisons between the difference image PT2 and the threshold 
level TH shown by the equations (4) and (6) and the equations (9) and (10), data of the threshold level TH is 
45 written from the CPU 10 into the first latch circuit 36 and the data of the difference image PT2 in the 
second latch circuit 37. By subtracting one fron the other of these data in the adder/subtractor circuit 38, a 
logical level of the boundary excluded image PT3 (Fig. 4) is judged. 

A judge portion 41 is also corinected to the video data bus 14. The judge portion 41 includes a third, a 
fourth and a fifth latch circuits 42, 43 and 44. The third latch circuit 42 stores luminance data of the 
so workpiece, the fourth latch circuit 43 stores the data of the boundary excluded image PT3 and the fifth latch 
circuit 44 stores data concerning the excluded area from the detection exclusion area memory 30. 

Data DATA1, DATA2 and DATA3 stored in the respective latch circuits 42, 43 and 44 are supplied to a 
judge circuit 45 by which an acceptability of the workpiece is judged, together with a judge parameter data 
DATA4 set by the CPU 10 in a judge parameter latch circuit 46, a resultant data DATA of this judgement is 
55 provided as a judge output of the pattern recognition device of this embodiment. 

The pattern recognition device shown in Fig. 1 operates to judge whether or not the workpiece 2 has 
minute defect or defects according to a program shown in Fig. 6. 

In Fig. 6, the CPU 10 starts to operate in the step SP1 at which the master -workpiece 2 has been set 
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with respect to the TV camera 3. In the step SP2, the CPU 10 picks up a frame of image data in the input 
image memory 5 on the basis of a video signal VD of the master workpiece. Simultaneously, the CPU 10 
sets the address correction data AX = 0 and AY = 0 in the address correction circuit 13,-so that the 
address data ADR in the timing control portion 6 is inputted to the input image memory 5 as the corrected 
address data ADRX upon which the data of the original master image PTO (Fig. 1) is set in the input image 
memory 5. 

Then, in the step SP3. the CPU 10 clears an image data in the working image memory 25 and, in the 
step SP4, it sets the correction data D AM composed of the shift data AX and A Y mentioned with respect to 
the equations (9) and (10) in the address correction circuit 13 pixel by pixel sequentially. 

Thus, in the step SP5, data of an image shifted in X and Y directions by A X and AY at a timing of the 
address data ADR from the timing control portion 6 are readout pixel by pixel and, after added by the 
image data in the working image memory 25 of the data operation portion 35, rewritten in the working 
image memory 25. 

Thereafter, in the step SP6, the CPU 10 judges whether or not the integration is completed and, if not, 
the operation is returned to the step SP4 to execute the operations in steps SP4 and SP5 for a next pixel. In 
the similar manner, the CPU 10 repeats the operations in the steps SP4, SP5, SP6 to SP4 for luminance 
data of respective pixels of respective shifted images i = -n- +n. j = -m- +m and the integration for all of 
the shifted images PT1 to PT4 completes. Thus, the integrated image data for obtaining the overlapped 
image PT1 mentioned with respect to Fig. 2 is stored in the working image memory 25. 

Under the circumstances, the judgement in the step SP6 becomes yes and then the operation is shifted 
to the step SP7 in which the data in the working image memory 25 is transferred to the latch circuit 37 of 
the operation portion 35 and, after divided in the shifter 39, returned to the working image memory 25. 
Thus, the overlapped image PT1 shown in Fig. 2 is stored in the working image memory 25. 

Thereafter, the operation is shifted to the step SP8 in which the data in the working image memory 25 
is readout to the latch circuit 36 of the data operation portion 35 pixel by pixel and image data of 
corresponding pixels thereto are readout from the input mage memory 5 to the latch circuit 36 and, after the 
division operation shown by the equation (2), an absolute value of a result thereof is returned to the working 
image memory 25. Therefore, the image data of the differential image PT2 (Fig. 3) is obtained in the 
working image memory 25. 

Then, the operation is shifted to the step SP9 in which the data of the threshold level TH is set in the 
latch circuit 36 and then, in the step SP10, the image data of the difference image PT2 stored in the 
working image memory 25 is compared with the- data of the threshold level TH in the latch circuit 36 pixel 
by pixel and data of 1 bit corresponding to sign thereof is written in the boundary data memory 29 which 
now stores the image data of the boundary excluded image PT3 (Fig. 4). 

Now, the setting and processing of the image PT3 representing the detection exclusion area PT31 to be 
excluded in judging the boundary of the master is completed. 

Thereafter, the operation is shifted to the step SP11 in which the image data in the input image memory 
5 is transferred to the working image memory 25 for subsequent judgement of the acceptability of 
workpieces. 

Then, the operation is shifted to the step SP12 in which the image data in the input image memory 5 is 
readout to the latch circuit 37 and subjected to an addition and a subtraction with respect to the data of the 
threshold level TH set therein, results being stored in the working image memories 26 and 27 as an upper 
limit data and a lower limit data, respectively. 

Thus, the processing of the image data of the master is completed and the operation of the CPU 10 is 
shifted to a processing step for judging acceptability of workpieces set with respect to the TV camera 3 one 
by one on the basis of the data of the master. After an image data based on a video signal VD of the 
workpiece is stored in the input image memory 5 in the step SP13, the operation is shifted to the step SP14 
in which the image data in the memory 5 is transferred to the latch circuit 36 and the image data in the 
working image memory 25 is transferred to the latch circuit 37. These data are subjected to a subtraction 
operation in the adder/subtractor circuit 38, an absolute value of a result of which is written in the 

comparison data memory 28. 

Since the working image memory 25 stores the image data of the master and the input image memory 
5 stores the image data of the workpiece to be judged, the comparison data memory 28 stores difference 
data in luminance between respective pixels of the master image and the workpiece image. 

The difference data is also considered for the boundary regions of the master image and the workpiece 
image and is stored in the comparison data memory 28 as data representing a difference in video signals 
VD obtained from the TV camera 3. Therefore, when a position of the workpiece is displaced from the 
position of the master, he luminance difference becomes large since the boundary regions thereof are 
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displaced from each other. When a size of the workpiece is different from that of the master, it is stored in 
the comparison data memory 28 as data representing a large luminance difference since the boundary 
regions thereof displaced largely from each other. 

Such difference data of the boundary region is excluded in judging the acceptability of the workpiece in 
the step SP15. 

That is, in the step SP15, the data of the threshold level TH is latched in the latch circuit 36 and the 
image data in the comparison data memory 28 is latched in the latch circuit 37 pixel by pixel. These 
latched data are compared in the adder/subtractor circuit 38 and a comparison data of one bit represented 
by a sign of an output of the circuit 38 is latched in a third latch circuit 42 of the judge operation portion 41. 

At the same time, the CPU 10 latches the data of the boundary excluded image PT3 in the boundary 
data memory 29 in a fourth latch circuit 43 and latches the data in the detection exclusion area data 
memory 30 in a fifth latch circuit 44. 

Thus, the judge circuit 45 responds to the data DATA1, DATA2 and OATA3 from the latch circuits 42, 
43 and 44 to provide a judge data OG which is logical "1'' when DATA1 = "IVi DATA2 = "0" and n 
DATA3 = "0", that is, when luminance of a pixel of the workpiece is the same as that of the master, the 
pixel is included in the detection region PT32 (Fig. 4) and the pixel is not included in the detection 
exclusion region T31, and logical "0" when DATA1 = "(T.u DATA2 = M 1" or u DATA3 = "1", that is, 
when luminance of the pixel of the workpiece is different from that of the master, the pixel belongs to the 
detection exclusion region PT31 or the pixel belongs to the detection exclusion region PT31. 

The judge circuit 45 counts the number of pixels for which the judge data OG obtained is logical "1" 
and the CPU 10 provides a judgement data DATA5 indicative of a negative decision in the step SP16 when 
the number of the pixels is larger than the value indicated by the .data DATA4 set in a judge parameter latch 
circuit 46 by the CPU 10. Then, the operation is shifted to the step SP17 in which a processing for defect 
workpiece is performed and, then, returned to the step SP13. 

When the counted pixel number is smaller than the content of the data DATA4, the CPU 10 provides 
the data OATA5 indicative a positive decision and the operation is shifted to the step SP18 in which a 
processing for acceptable workpiece and, then, returned to the step SP13. 

According to the embodiment mentioned above, the data which represents the border line of the region 
to be observed and which contains large luminance difference compared with that to be detected is 
excluded in judgement when it belongs to the predetermined region PT31 (Fig. 4). Therefore, it is possible 
to detect minute defect or defects reliably even when the position of the workpiece is displaced from that of 
themaster or the border line of the workpiece is not coincident with that of the master due to a difference in 
size therebetween. 

In another embodiment of the present invention, the equation (8) is replaced by the following equation, 
m m 

j=-m i--n f<X + i0 ** ' y + i'*Y > 

*T3c7y) ~ '*° ... (13 ) 

2m ° 2n 

That is, although, in the equation (8), the case where i = 0 and j = 0 is considered to obtain the mean 
luminance of each pixel, the case is excluded in the equation (13). 



Claims 

1 . A defect recognition device for a workpiece, comprising means for obtaining a first video image data 
of said workpiece, means for obtaining a second video image data of a master, means for comparing said 
first video image data with said second video image data, means for excluding image data portions of said 
first and said second video image data corresponding to border regions thereof and judging means 
responsive to an output of said excluding means for judging whether said workpiece includes defect or 
defects. 

2. The defect recognition device as claimed in claim 1 , wherein said comparing means includes means 
for shifting an original video image represented by said second video image data in predetermined 
directions by a predetermined distance to produce a plurality of shifted images, means for forming an 
overlap image data representing an overlap image formed by overlapping said original image with shifted 
images and means for forming a difference image daa by differentiating said overlap image data and said 
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original image data and wherein said excluding means comprises means for producing an image data 
including data of a region to be excluded and a region to be observed by comparing said difference image 
data with a predetermined threshold value and means for excluding portions of said first and said second 
image data corresponding to said data of said region to be excluded from an output -of said producing 
5 means. 

3 The defect recognition device as claimed in claim 2, wherein said judging means judges a 
coincidence of said first video image data with said second video image data when luminances of 
respective pixels of said first and said second video image data are equal to each other and said pixels are 
within not said region to be excluded but said region to be observed. 
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